VIDEO AND DIGITAL MULTIMEDIA AGGREGATOR 
CONTENT CODING AND FORMATTING 

Related Applications 

This application is a continuation-in-part of patent application serial number 
09/920,723, entitled "Video and Digital Multimedia Aggregator," filed on August 3, 2001 
and patent application serial number 09/920,615, entitled "Video and Digital Multimedia 
Aggregator Remote Content Crawler," filed on August 3, 2001. These patent applications 
are hereby incorporated by reference. 

The following U.S. Patents also are incorporated by reference: 

U.S. Patent 5,798,785, entitled "Terminal for Suggesting Programs Offered on a 
Television Program Delivery System;" 

U.S. Patent 5,986,690, entitled "Electronic Book Selection and Delivery System;" 

The following co-pending U.S. patent applications also are incorporated by reference: 

patent application serial number 07/991,074, filed December 9, 1992, entitled 
"Remote Control for Menu Driven Subscriber Access to Television Programming;" 

patent application serial number 08/906,469, filed August 5, 1997, entitled 
"Reprogrammable Terminal for Suggesting Programs Offered on a Television Program 
Delivery System;" 

patent application serial number 09/191,520, filed November 13, 1998, entitled 
"Digital Broadcast Program Ordering;" 

patent application serial number 09/289,957, filed April 13, 1999, entitled "Electronic 
Book Alternative Delivery Systems;" 

patent application serial number 09/289,956, filed April 13, 1999, entitled "Electronic 
Book Alternative Delivery Methods;" and 

patent application entitled "Video and Digital Multimedia Aggregator;" and patent 
application entitled "Video and Digital Multimedia Aggregator Remote Content Crawler," 
filed on date herein. 
Technical Field 

The technical field is television program and digital multimedia delivery systems that 
incorporate intelligent and flexible program search and delivery mechanisms. 
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1 Background 

2 Cable television delivery systems greatly expanded the array of programs available to 

3 television viewers over that provided by over-the-air broadcast systems. Subsequently, 

4 satellite television delivery systems, and in particular, digital satellite broadcast systems 

5 further expanded the viewing choices for consumers. In the near future, digital broadcast 

6 television systems will provide many more programming choices for consumers. 

7 In addition to television programming delivered through television program delivery 

8 systems, other programs and events may be sent to consumers. These other programs and 

9 events include streaming video sent over wired and unwired, narrowband to broadband 
m 10 services, digital audio programs, and other multimedia data. 

tfj 1 1 Unfortunately, customers are still limited in their television viewing choices by the 



St 12 local and regional nature of television delivery systems. For example, a broadcaster in 

luuf 

?i 13 Boston may provide programming of local interests to people in Massachusetts while a 

VJ 14 broadcaster in Seattle may provide different programming to people in the Seattle area. A 

» 15 person in Boston generally cannot access Seattle programming, other than programming that 

JIJ 16 is provided at a national level. 

^ 17 In addition to this local/regional market segregation, many other sources of 

Q 18 programming and events may not be available in a specific viewing area. These other 

: ^ 19 sources may include audio programming, streaming video, local or closed circuit television 

20 programming (e.g., education television programming provided by a state education 

2 1 department) and other programming. 

22 Even if certain programming is available in a local viewing area, a viewer may not be 

23 aware of its existence. This situation may be the result of a large array of available 

24 programming coupled with a limited program menu or guide. The program guide may be 

25 limited in that not all available programming can be listed, some programming changes occur 

26 that are not reflected in the program guide, and errors may exist in the program guide. In 

27 addition, the more comprehensive the program guide, the harder it is for the viewer to search 

28 and navigate the program guide to find a desired program. 

29 Summary 

30 The problems noted above are solved by the video and digital multimedia aggregator 

31 system and method described herein. Program content can be packaged and delivered by the 
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system, including video, television, radio, audio, multimedia, computer software, and 
electronic books, or any content that can be delivered in digital format. 

The aggregator comprises a request and results processing server, a search engine 
server coupled to the request and results processing server and a content acquisition server 
coupled to the request and results processing server. The request and results processing 
server receives a request for a program, the search engine server searches for the program 
and the content acquisition server acquires a program for delivery to the user. The request 
and results processing server includes a search request processor that receives information 
related to the user's search request and provides the information to a search results form 
builder that creates an electronic search request. The search request may be augmented by 
using a content search suggestion engine to add additional search terms and descriptions to 
the search request. The aggregator may also include a decoder that decodes program content 
and program metadata from remote sources for storage at the aggregator, and an encoder that 
encodes content metadata and programs for delivery to the user. The aggregator may also 
comprise one or more crawlers, such as a content crawler, to look for program content in the 
digital communications network. 

The search engine server searches at least a local content database. The local 
database comprises at least two file types. A content file includes a complete program 
content file. For example, the 1997 movie Titanic may exist in the local content database as 
a complete program content file. The complete program content file may also include a 
reference content or metadata that contains additional information related to the content. 
Such additional information in the reference content may include: a program description, 
including program rating, program description, video clips, program length, format (e.g., 4x3 
television or 16x9 movies), and other information; billing information and digital rights 
management information; viewing statistics, including number of times viewed, dates/times 
viewed, identity of users viewing the program; advertisement information to allow ads to be 
inserted during viewing of the program; and other information. 

The additional information in the reference file may be provided in whole or in part to 
the system users. For example, the aggregator may provide a program description and 
accompanying video clips to selected systems users. The reference file may also be used by 
the aggregator for system administration purposes. For example, billing and digital rights 
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1 management information may be used to collect appropriate fees from system users and to 

2 provide such collected fees to the entities owning rights in the content. 

3 A remote content crawler continually crawls the digital communication network 

4 looking for content to provide to the aggregator. The content provided to the aggregator may 

5 be stored in a form of an entire content file. For example, the content may include an entire 

6 movie, television program or electronic book. Alternatively, the content provided to the 

7 aggregator may be a reference to a content file that is stored at, or that will be available at 

8 one of the remote locations. For example, the content may be a reference to a future, 

9 scheduled live sports event that will be made available to system users. The sports event 
10 may be provided for a one time fee, as part of a sports package, for which a fee is collected, 

y| 11 or as a free event. In the examples discussed above, the content may be stored at the 

12 aggregator, and may subsequently be provided to system users. For the example of the live 

W 13 sports event, the aggregator may store the live sports event and may then provide the sports 

™ 14 event as a replay, in addition to facilitating live viewing of the sports event. 

" 4 15 To ensure the content is delivered to the correct addressee, in a format that is 

y* 16 compatible with the addressee's equipment (i.e., hardware and software), the aggregator may 

17 include one or more content coders, decoders and formatters. In an embodiment, the coding, 

C3 18 decoding and formatting functions may be executed by separate components, which may be 

13 ' 

U 19 embodied as hardware or software, or a combination of hardware and software. In another 

20 embodiment, two or more of the functions may be executed by a single device, comprising 

21 hardware and/or software. 

22 In an embodiment, the formatting, decoding and coding functions are executed by a 

23 coder-decoder (codec) formatter. The codec formatter may comprise means for receiving a 

24 coding and formatting request, means for analyzing parameters contained in the coding and 

25 formatting request, means for decoding, formatting and coding target content, means for 

26 configuring the decoding, formatting and coding means, and means for routing coded target 

27 output content to one or more target addresses. 

28 In another embodiment, the codec formatter may include means for processing 

29 auxiliary service requests. Such auxiliary service requests may be included in the coding and 

30 formatting request, or may be separately supplied. The auxiliary services may include closed 

31 captioning, descriptive video narration, alternative language audio, content rating, critical 
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review information, device control and commands, future content schedules, advertising, 
targeted advertising, text and data services, interactive services, and content metadata. The 
means for processing auxiliary services includes means for analyzing the auxiliary service 
requests, means for configuring one or more auxiliary services processing means to supply 
the requested auxiliary services, and means for outputting the requested auxiliary services. 
The outputted auxiliary services may be combined with the coded target output content. 
Description of the Drawings 

The detailed description will refer to the following drawings in which like numerals 
refer to like elements, and in which: 

Figure 1 is a block diagram of primary components of a content search, packaging, 
and delivery system; 

Figure 2 is a block diagram of the components of the content search, packaging, and 
delivery system configured to deliver content to a set top terminal; 

Figure 3 is a block diagram of the components of the content search, packaging, and 
delivery system configured to deliver content to a personal computer terminal; 

Figure 4 is a schematic of the components of the content search, packaging, and 
delivery system showing subsystems of an aggregator, a user terminal, and a remote content 
server; 

Figure 5 is a schematic of subsystems of a content acquisition server and components 
with which the content acquisition server interacts as part of a content search, packaging, and 
delivery system; 

Figure 6 is a schematic of subsystems of a content delivery server and components 
with which the content delivery server interacts as part of the content search, packaging, and 
delivery system; 

Figure 7 is a schematic of subsystems of a codec and content formatter and 
components with which the codec and content formatter interacts as part of the content 
search, packaging, and delivery system; 

Figure 8 is a diagram of components of a formatting codec used with the codec and 
content formatter of Figure 7; 

Figure 9 shows a flowchart describing a process of decoding, formatting, and coding 
source content and outputting target content; and 
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Figure 10 shows a flowchart describing a method of processing auxiliary services that 
may accompany source and/or target content. 
DETAILED DESCRIPTION 

Figure 1 is a block diagram of a content search, packaging, and delivery system 200. 
The content to be packaged and delivered by the system 200 includes video, television, radio, 
audio, multimedia, computer software and electronic books. Components of the system 200 
include an aggregator 201 and a user terminal 202, which are connected using a wide area 
distribution system 203. Other components are remote content servers 204 that exchange 
data with the aggregator 201 using a wide area network/Internet 205 connection. The user 
terminal 202 may incorporate a video display system 207, an audio playout system 208, an 
audio/video recording system 209, user input devices 214, an electronic book reader 230, and 
a connection to a home network 229 to interact with other devices in the user's home 
environment. Alternatively, one or more or all of the video display system 207, the audio 
playout system 208, the audio/video recording system 209, and the electronic book reader 
230 maybe separate components that are coupled to the user terminal 202. 

The system 200 allows a user to enter search parameters, such as keywords and 
category headings, and have the aggregator 201 use these parameters to locate, package, and 
deliver content to the user terminal 202 from numerous sources. The requests and content 
deliveries can be sent over communications links including, but not limited to, telephone 
lines, coaxial cable, fiber-optic cable, wireless connections, wide area networks, the Internet, 
and other communication media collectively represented by the wide area distribution system 
203. The numerous sources of content are shown in Figure 4 and include, but are not limited 
to, an aggregator local storage 254, local streaming sources 262, remote content storage 258, 
and remote streaming sources 259. In an embodiment, the local streaming sources 262 are 
comprised of the audio and video channels being delivered using a cable headend system that 
may house the aggregator 201 . 

The system 200 will take a user's search request and may perform a search of 
virtually every active and scheduled radio and television channel in the world, as well as 
archived sources of video and audio programming such as movies and recorded audio 
sources and sources of other multimedia, software, and electronic book content. In an 
embodiment, the system 200 will also search Internet Web sites and other online databases. 
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1 The user will then be able to select programming or other content for download based on the 

2 search results. In an embodiment, the download, or delivery, process can be fulfilled by 

3 making the content available on a specific channel of a cable television system, or by 

4 transmitting the content using a digital communications protocol, such as the Internet 

5 standard TCP/IP, for example. In addition, the content search, packaging, and delivery 

6 system 200 is capable of formulating and presenting a list of suggested content based on an 

7 analysis of the user's current search parameters, stored information about previous searches 

8 and previously selected content downloads and other user-specific or related information. 

9 The system 200 is also capable of notifying a user prior to the start time of selected 
10 programming and availability of other content using such notification as an electronic mail 

y§ 11 message and/or an on-screen message indicating that the scheduled program will be 

12 broadcast at a specified time. The system 200 may support one or more digital rights 

W 13 management (DRM) systems to track the usage and copyrights rights associated with 

qft 14 downloaded content and bill the user's account as appropriate and provide any license and 

^ 15 usage fees to the content provider. The system 200 may implement a users' privacy 

H 16 protection scheme allowing users to control what information is gathered, limit what is done 

jj 1 7 with that information, and review and delete the user's profile if desired. 

18 An overview of an embodiment of the system 200 is shown in Figure 2 where the 

|# 19 user terminal 202 is a set top terminal 206 that communicates with the aggregator 201 

20 through a cable television system headend 210, thereby making use of the cable television 

21 system headend 210 high bandwidth concatenated cable system 216. Further, the set top 

22 terminal 206 system may include a tuner 228, a demultiplexer 231, the video display system 

23 207, the audio playout system 208, the audio/video recording system 209, user input devices 

24 21 4, an electronic book reader 230, and a connection to the user's home network 229. These 

25 components are used to tune, select, view, listen to, and store audio and video programming 

26 and other content delivered by the search, packaging, and delivery system 200. Figure 2 also 

27 shows a communications path from one or more remote content servers 204 through the wide 

28 area network/Internet 205 directly to the set top terminal 206, which bypasses the aggregator 

29 201 and cable television system headend 210. This path may be used in the case where the 

30 requested content is available in the required format from the remote content server 204 and 

3 1 is authorized for direct delivery to the user. In an alternative embodiment, the aggregator 
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1 201 is collocated with the cable television system headend 210 that is acting as the wide area 

2 distribution system 203 as is the case when a cable television system is also serving as the 

3 user's Internet service provider. 

4 An overview of another embodiment of the system 200 is shown in Figure 3, where 

5 the user terminal 202 is a personal computer terminal 21 1 that communicates with the 

6 aggregator 201 through an Internet service provider/cable television system headend 215. In 

7 this case, the content may be delivered by a cable headend that is operating as an Internet 

8 service provider (ISP). The personal computer terminal 211 also may include the video 

9 display system 207, the audio playout system 208, the audio/video recording system 209, 

10 user input devices 214, the electronic book reader 230, and the connection to the home 

11 network 229. A communications path also exists from one or more remote content servers 
*p 12 204 through the wide area network/Internet 205 directly to the personal computer terminal 
y 13 211, which bypasses the aggregator 201 and the Internet service provider/cable television 
M 14 system headend 215 in the case where requested content is in the required format and is 
\l 1 5 authorized for direct delivery to the user. 

U 16 The user can receive video and audio programs (i.e., the content) in a number of 

U 17 different ways. First, the tuner 228 in the user terminal 202, shown in Figure 2, can simply 

O 18 tune to the desired program, or the demultiplexer 231 can select a program from a digital 

P 19 multiplex, thereby displaying the program at the appropriate time. However, the desired 

20 program may not be broadcast by the user's cable television system headend 210, or the user 

21 would like to watch a program supplied by the local headend, but not subscribed to by the 

22 user, e.g., a boxing match broadcast by a premium channel that the user does not ordinarily 

23 receive. In these examples, the program can be delivered to the user by the aggregator 201 

24 using the telephone lines, fiber-optic transmission lines, or other communication media, or 

25 using the cable television system headend 210. In an embodiment, the aggregator 201 can 

26 supply the desired program to the user by pulling programs from program delivery systems in 

27 the United States and abroad and delivering the program to system users. Therefore, a user 

28 can have access to programs outside the user's immediate viewing area. 

29 Figure 4 shows the system 200 and system components in more detail. At a user's 

30 location, the user terminal 202 includes the tuner 228, the demultiplexer 23 1, a user terminal 

31 processor 227, user local storage 212 and user local database 213. Coupled to the user 
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1 terminal 202 may be user input devices 214, the video display system 207, the audio playout 

2 system 208, the audio/video recording system 209, an electronic book reader 230, and a 

3 connection to the home network 229. 

4 The user terminal 202 is coupled through the wide area distribution system 203 to the 

5 aggregator 201 and further through the wide area network/Internet 205 to remote program 

6 sources. The remote program sources include remote streaming sources 259 and remote 

7 central storage 258. The remote program sources also include remote databases 261 and, 

8 through the remote content server 204, a remote server database 260. 

9 The aggregator 201 may include a communications server 250 that communicates 
f;i 10 with the user terminal 202 through the wide area distribution system 203. The 
V 1 1 communications server 250 receives inputs from a request and results processing server 300, 
%l 12 a content delivery server 450 and a system administrator 500. The content delivery server 
SS 13 450 receives inputs from a coder and content formatter 253 and a content acquisition server 

14 400. The content delivery server 450 also accesses an aggregator local storage 254 and local 

. 15 streaming sources 262. Finally, the content delivery server 450 provides an output to the 

16 system administrator 500. 

if 17 The coder and coder formatter 253 receives inputs from the content acquisition server 

q 18 400, the aggregator local storage 254 and local streaming sources 262. The system 

rr ' 19 administrator 500 receives inputs from the content and delivery server 450, and 

20 communicates with the content acquisition server 400, the request and results processing 

21 server 300, a search engine server 350 and aggregator archives 255. A decoder and content 

22 formatter 252 is coupled to the content acquisition server 400. Finally, a network gateway 

23 251 couples components of the aggregator 201 with the remote content server 204 through 

24 the wide area network/Internet 205. 

25 Programs received at the aggregator 201 may be input to the formatter 253. The 

26 formatter 253 reformats all input content into a format that is readily received by all user 

27 terminals 202 operably connected to the delivery system 200 (not shown in Figure 4). In 

28 addition, the formatter 253 can store full or partial copies of content in compressed form in 

29 the aggregator local storage 254. Furthermore, the aggregator 201 can provide real-time 

30 delivery of certain content, such as a boxing match. In an embodiment, if a large number of 

31 users want a particular live program, then the cable television system headend 210 can 
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broadcast the program on a particular channel available to all the requestors instead of 
broadcasting the program to each individual user over the wide area distribution system 203. 
See U.S. Patent Application serial number 09/191,520, entitled Digital Broadcast Program 
Ordering, hereby incorporated by reference, for additional details of broadcast program 
delivery. 

The aggregator 201 can also implement a screening process for limiting the number 
of programs captured to those programs with a viewing audience above a predetermined 
threshold. In effect, the aggregator 201 contains a filter that will pass only programs meeting 
the predetermined selection criteria. The filter may include programming that screens the 
content to reject specific items, such as adult content, for example. 

The system administrator 500 records what fees should be paid and to whom. For 
example, the aggregator 201 will determine to whom any copyright or other fees should be 
paid when a program is broadcast. 

The user terminal 202 may be a television, a set top terminal 206, a personal 
computer terminal 211 (not shown), or any device capable of receiving digital or analog data, 
or some combination thereof. The user terminal 202 is equipped with the user input devices 
214 that communicate search criteria to the system 200 as well as navigate through the user 
terminal menu system and control the user terminal's other various functions. The user local 
storage 212 is used to store and archive content onto one or more removable and/or non- 
removable storage devices or media for later access. Removable storage media may include, 
but is not limited to, magnetic tape, magnetic disks, optical disks and modules, and electronic 
memory cartridges. The user local database 213 is the repository of all relevant information 
about a user's profile and account. This information includes, but is not limited to, user 
name, password, personal information that the user has authorized for storage, billing 
information, other users allowed access to the account, past search criteria, past content 
download information, and library information about stored content. As a consumer 
protection, the user terminal 202 may enable the account user to view the information stored 
in the user local database 213 and modify certain data fields and select which data fields may 
be reported to a main system database within the aggregator 201. Certain fields including, 
but not limited to, account numbers and billing information may not be allowed this level of 
user access. 
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1 The user terminal processor 227 may include a central processing unit and all 

2 associated hardware, software, and firmware modules to perform all operations within the 

3 user terminal 202. These operations include, but are not limited to, managing 

4 communications with the aggregator 201 and other networked devices, processing search and 

5 download requests, displaying search and download results, managing and controlling 

6 communications with the user local storage 212 and the user local database 213, responding 

7 to user interaction with presentation of graphical user interface (GUI) menus, playing out 

8 selected programming content using various audio and video output devices, implementing 

9 the user's part of the digital rights management schema, and administering the user's account 
10 and billing. The tuner 228 and the demultiplexer 231 are used to select an audio/video 

, Jj 1 1 channel for playout from the channels available on the cable television system 2 1 6. 

^12 In an embodiment, the user terminal 202 may incorporate selected features of the 

hi 13 aggregator 201. For example, the user terminal 202 may include a small metadata crawler, 

m 1 4 an aggregator, and program content and program metadata storage. 

15 The user terminal 202 communicates with the aggregator 201 using the wide area 

1*4 16 distribution system 203. Within the aggregator 201, the communications server 250 acts as 

2 17 the interface point to the wide area distribution system 203 for the purpose of managing 

W 18 primary communications to system users. The communications server 250 routes incoming 

jy, 19 user requests and associated user information to the request and results processing server 

20 300, routes search results and content downloads through the wide area distribution system 

21 203 to end users, and may route billing information from a customer billing server to the end 

22 users. The request and results processing server 300 performs the basic processing and 

23 routing related to user search requests, content download requests, administrative information 

24 requests, search results, related content suggestions, and programming notification. 

25 The content acquisition server 400, as shown in detail in Figure 5 receives content 

26 download requests from the request and results processing server 300. The content 

27 acquisition server 400 includes a content request processor and router 401 and a remote 

28 content download processor 402. A download request is made by the user for content 

29 selections from sources including, but not limited to, earlier programming searches, recurring 

30 scheduled events, an electronic program guide, lists of electronic books and computer 

31 software, advertisements, promotions, and affiliated Internet websites. In general, a 
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1 download request will be accompanied by data indicating the source of the content and 

2 whether the content is local or remote. Requests for content that is available only from a 

3 remote site may be routed to the network gateway 251 and then to the appropriate remote 

4 source. Remote sources include, but are not limited to, the remote content storage 258 and 

5 the remote streaming sources 259 shown in Figure 4. The remote content server 204 verifies 

6 the request and returns the content through the wide area network/Internet 205. If the 

7 programming content is in the appropriate format and is authorized for direct delivery to the 

8 user, the content may be routed through the wide area network/Internet 205 and the wide area 

9 distribution system 203 directly to the user terminal 202. During this transfer, administrative 
q 1 0 data pertaining to billing and verification of delivery may be sent back to the aggregator 20 1 . 
I ' 11 Programming that is not formatted and authorized for direct delivery from a remote 
§ j[ 12 source may be transmitted through the network gateway 251 to the remote content download 
Q 13 processor 402 contained within the content acquisition server 400. The remote content 

14 download processor 402 buffers or caches the programming content while managing the 

J% . 15 download connection to the remote source. The programming content is then routed to the 

Q I 6 decoder and content formatter 252, which may be capable of decoding various industry 

g 17 formats and compression schemes and may reformat and encode the decoded data and 

jd 18 associated metadata into one or more preferred content formats for delivery and for local 

19 storage. The processed programming content, along with user identification and routing 

20 data, is then routed to the content delivery server 450, which then manages the delivery of the 

21 content through the communications server 250 and wide area distribution system 203 to the 

22 user terminal 202. The content delivery server 450 is shown in detail in Figure 6. The 

23 content acquisition server 400 is described in more detail in co-pending patent application 

24 entitled "Video and Digital Multimedia Aggregator," filed on August 3, 2001, which is 

25 incorporated by reference. 

26 Requests for programming available from a local source are routed directly to a local 

27 content request processor 451 within the content delivery server 450. The local content 

28 request processor 451 initiates delivery of content from local sources including, but not 

29 limited to, the aggregator local storage 254 and aggregator local streaming sources 262, and 

30 routes the content to the coder and file formatter 253. In an embodiment, the local streaming 

31 sources 262 can include any of the cable television channels available at the cable television 



12164 



1 system headend 210 housing the aggregator 201. If the programming content is not already 

2 in the user-requested format and coding scheme, the coder and content formatter 253 (see 

3 Figure 4) performs content formatting and coding of the programming for delivery to the 

4 user. The content is then forwarded to a content delivery processor 453, which manages the 

5 delivery of the content through the communications server 250 and wide area distribution 

6 system 203 to the user terminal 202. 

7 If the programming being delivered from a remote content server 204 will be stored 

8 to the aggregator local storage 254, the programming metadata is analyzed to determine if the 

9 content is in an acceptable format for local storage. The aggregator 201 may be configured 
„ 10 to store content in one or more specific formats that will balance the highest quality of 
tf? 11 programming content to be delivered to the users versus available storage space. In an 
%| 12 embodiment, one or more of the selected storage formats will make the most efficient use of 
=ji~ 13 the aggregator local storage 254 resources as well as support high-speed delivery to system 
fl 14 users. If the content acquisition server 400 determines that the content does not meet the 
5 15 format requirements for local storage, the content stream and/or contents are routed to the 
Pj 16 decoder and content formatter 252 (see Figure 4). The decoder and content formatter 252 
H- 17 decodes the incoming data and may extract digital and/or analog data representing the 
Q 18 content and any metadata associated with the content. The decoder and content formatter 
^ 19 252 then reformats the content into the required formats and coding schemes for local 

20 storage. The properly formatted programming content may then be stored to the aggregator 

21 local storage 254. 

22 Returning to Figure 5, the remote content download processor 402 analyzes the 

23 programming content's metadata and determines if the content is in the appropriate format 

24 and coding scheme for delivery to the user. The format and coding scheme for delivery may 

25 be different from that used for local storage in order to accommodate particular parameters of 

26 a user's download request. The user may request specific formats and coding schemes due to 

27 considerations including, but not limited to, the method of playback, the type of 

28 programming, whether the user intends to store the content, or the type and bandwidth of the 

29 connection the user terminal 202 has to the aggregator 201 or the remote content server 204. 

30 If the content does not meet the format criteria, the content stream and/or contents are routed 

31 to the decoder and content formatter 252 along with the required formatting and coding 
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parameters. The decoder and content formatter 252 decodes the incoming data and may 
extract digital and/or analog data representing the content and its metadata. The decoder and 
content formatter 252 then reformats the content into the required formats and coding 
schemes for delivery to the user. 

If the programming download request is to be fulfilled by the local aggregator 201, 
the local content request processor 451 of the content delivery server 450 (see Figure 6) 
analyzes the programming content's metadata and determines if the content is in the 
appropriate format and coding scheme for delivery to the user. If the format and coding 
scheme is not correct, the local content request processor 451 retrieves the programming 
content from the aggregator local storage 254 or the appropriate local streaming sources 262 
and routes the programming to the coder and content formatter 253 along with the required 
formatting and coding parameters. The coder and content formatter 253 then decodes the 
incoming data and may extract digital and/or analog data representing the content and its 
metadata. The coder and content formatter 253 then reformats the content into the required 
formats and coding schemes for delivery to the user. The correctly coded and formatted 
programming content file(s) and/or streams, whether directly from local sources or from the 
coder and content formatter 253, and routes this content to the content delivery server 450. 

The coder and content formatter 253 may employ digital compression techniques to 
increase existing transmission capacity. A number of digital compression algorithms 
currently exist or may be developed in the future that can achieve the resultant increase in 
capacity and improved signal quality desired for the system 200. For television and video 
content, algorithms generally use one or more of three basic digital compression techniques: 
(1) within-frame (intraframe) compression, (2) frame-to-frame (interframe) compression, and 
(3) within carrier compression or a combination of two or more techniques. The coder and 
content formatter 253 of the content search, packaging, and delivery system 200 will be able 
to use any one or combination of two or more of these techniques in addition to being able to 
have its capabilities modularly expanded to include any emerging and future techniques that 
are determined to be desirable. In an embodiment, although a single digital compression 
standard may be used for the delivery system 200, different levels of compression and/or 
digital coding can be utilized when delivering programs to a user requesting a particular 
program. For example, if the program is sent out using an HDTV format, then less 
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1 compression is used as opposed to sending out the program using a standard video format. 

2 The reason is that the HDTV format requires more digital data per portion of content. Since 

3 the HDTV format requires more digital data, the HDTV format will also require more 

4 bandwidth to transmit, possibly increasing the cost of delivery. Therefore, the quality of the 

5 video delivered may be a variable in the fees charged to users of the system 200. 

6 The decoder and content formatter 252 and the coder and content formatter 253 may 

7 be similar or identical in functionality, software configuration, and/or hardware 

8 configuration. In an embodiment, the decoder and content formatter 252 and the coder and 

9 content formatter 253 may be embodied by a single subsystem represented by a codec and 
10 content formatter 270, which is shown in detail in Figure 7. In an alternate embodiment, the 

% 11 decoder and content formatter 252 and the coder and content formatter 253 may each 

© 12 comprise one or more separate codec and content formatters 270. The codec and content 

y 13 formatter 270 includes a codec and content formatter processor 271 that may communicate 

:J 14 with and control other components of the codec and content formatter 270. In particular, the 

%!. 15 formatter processor 271 communicates with one or more content routers 272, one or more 

16 formatting codecs 274, and one or more auxiliary services processors 276. The formatter 

17 processor 271 communicates with one or more content routers 272 in order to transmit 
Q 18 routing connection parameters and paths for the transfer of content data between various 
y, 19 source and target subsystems. The formatter processor 271 communicates with one or more 

20 formatting codecs 274 to identify and manage available codec processing resources, 

21 configure codec hardware and/or software parameters for decoding, formatting, and coding 

22 operations, and to transmit routing information and coding and formatting parameters 

23 necessary for the formatting codecs 274 to decode, format, and code content from various 

24 sources. The formatter processor 271 also communicates with one or more auxiliary services 

25 processors 276 to allocate resources and transmit parameters associated with requests for 

26 auxiliary services processing. Auxiliary services may include closed captioning, descriptive 

27 video services, advertising, interactive services, and other data services. The formatter 

28 processor 271 may also communicate with other components of the content search, 

29 packaging, and delivery system 200 including the content delivery server 450, the content 

30 acquisition server 400, the aggregator local storage 254, and the local streaming sources 262 

31 (see Figure 4). Communications with these other systems includes receiving coding and 
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1 formatting requests, transmitting content routing data, and managing content flow between 

2 various content sources and one or more components of the codec and content formatter 270. 

3 When the remote content download processor 402 within the content acquisition 

4 server 400 and/or the content delivery processor 453 within the content delivery server 450 

5 determine that a particular content entity needs to be decoded, formatted, reformatted, and/or 

6 coded, the respective processors (402, 453) transmit a coding and formatting request to the 

7 codec and content formatter 270. Coding and formatting requests convey data to the codec 

8 and content formatter 270 characterizing the input source content and the desired output 

9 target content. Data conveyed about the source and target content may include parameters 
m 10 such as physical and/or logical addresses, coding and compression parameters, format 
y3 11 descriptions, content sizes and/or lengths, description or location of auxiliary services 
u 12 combined with the source content, description or location of auxiliary services to be 
JJ; 13 combined with the target content, and other metadata elements that may be required for 
ft 14 coding, formatting, and routing. The codec and content formatter 270 may then use these 
s * 15 parameters to identify available resources, establish content routing paths, configure 
!J 16 formatting codecs 274, configure auxiliary services processors 276, and perform the required 
I** 17 coding and formatting operations to fulfill the coding and formatting request. In an 
g 18 embodiment, a coding and formatting request may identify one or more target content coding 
^ 19 and formatting schemes, as in the case where one type of formatting may be applied to the 

20 target content for storage and another type of formatting may be applied to the target content 

21 for delivery to system users, which may require the codec and content formatter 270 to 

22 configure and control one or more of the formatting codecs 274 and auxiliary services 

23 processors 276. 

24 The codec and content formatter 270 may use one or more of the content routers 272 

25 to dynamically establish logical and/or physical circuit connections for transmission of 

26 content data between the various subsystems shown in Figure 7. A content router 272 may 

27 comprise one or more hardware systems and one or more software modules that may operate 

28 under the direction of the formatter processor 271 to establish and manage the 

29 interconnections between subsystems required to transfer content in support of one or more 

30 coding and formatting operations. In Figure 7, several of the interconnections between 

31 subsystems are shown as two connection lines and a series of dots between the two lines. 
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1 This convention is used to indicate that at any time there may exist one or more connections 

2 or circuits between one or more of the connected subsystems and that between two connected 

3 subsystems the number of connections may vary. Each collection of connections or circuits 

4 may comprise one or more logical and/or one or more physical connections between software 

5 and/or hardware modules. Physical routing configurations, such as may be required for the 

6 transfer of analog content, may be achieved using one or more industry standard and/or 

7 custom components such as circuit switching devices, matrix switches, distribution 

8 amplifiers, signal splitters, input and output ports, and patch bays. Logical routing, such as 

9 may be required for the transfer of digital content, may be achieved using one or more digital 
^ 10 packet transport protocols that may operate on one or more local area networks supported by 
Ct 1 1 the system administrator 500 as part of the content search, packaging, and delivery system 

12 200. In an embodiment, the functionality of the content routers 272 may be implemented by 

^ 13 one or more subsystems and may be functionally and/or physically external to the codec and 

: b'.D.:? 

ft 14 content formatter 270 and may be a subsystem (not shown in Figure 4) of the aggregator 20 L 

^ T ■ 1 5 In another embodiment, the functionality of the content routers 272 may be implemented by 

H 16 one or more subsystems and may be functionally and/or physically external to the aggregator 

U 17 201 so that the aggregator 201 and its subsystems may connect to the content routers 272, or 

55 18 equivalent systems, through external interfaces (not shown in Figure 4). The number of 

H- 19 connections and the logical and physical routing paths of such connections both within the 

20 codec and file formatter 270 and external to the codec and file formatter 270 may be changed 

21 as required to accommodate a smaller or larger number of content pathways and to 

22 accommodate various source and destination combinations required at any given time. These 

23 pathways may be created or modified on a demand basis by the formatter processor 271 and 

24 may be modified prior to a coding and formatting operation and/or during a coding and 

25 formatting operation as may be required. 

26 Content that is stored locally by the system 200 and/or acquired remotely may exist in 

27 one or more of numerous standardized and/or proprietary formats with various coding and 

28 compression schemes applied and may need to be decoded, reformatted, and/or recoded to 

29 satisfy local storage requirements and requirements of delivery to one or more system users. 

30 The content decoding, formatting, and/or coding functions may be performed by one or more 

3 1 of the formatting codecs 274 as shown in Figure 7. Decoding of a content file or stream may 
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comprise several processing operations that may include performing a packet or frame 
alignment, reading and caching content metadata, reading embedded time code information, 
reading and caching auxiliary services data, and recovering and caching content payload data 
in digital form and/or analog baseband form. Formatting or reformatting of content payload 
data may comprise several processing operations that may include analog-to-digital 
conversion, digital-to-analog conversion, packetizing or framing digital data representing 
content into one or more standard and/or proprietary formats, translating digital data 
representing content from one or more standard and/or proprietary formats into one or more 
other standard and/or proprietary formats, modifications to existing auxiliary services that 
may be packaged with the content, addition of auxiliary services such as closed captioning 
and/or advertising, frame rate translation of video content, video aspect ratio format 
conversion (e.g., 4x3 television or 16x9 movies), and any required audio and/or video 
processing. Coding of a content file or stream may comprise several processing operations 
that may include applying varying levels of compression and various schemes of error 
correction coding to one or more content files and/or signals. 

The codec and content formatter 270 may contain one or more banks of one or more 
formatting codecs 274 that may comprise one or more separate software and/or hardware 
modules. The hardware and/or software modules may exist as a flexible group of resources 
that may be accessed on a demand basis and may be dynamically assigned and configured to 
accommodate virtually any content decoding, formatting, and coding operation. When the 
formatter processor 271 receives a coding and formatting task request from the content 
acquisition server 400 or the content delivery server 450, the formatter processor 271 may 
poll the group of formatting codec 274 resources, identify available resources required for the 
requested task, configure those resources to perform the requested task thereby configuring a 
formatting codec 274, and transmit the required decoding, formatting, and coding parameters 
as well as routing information to the newly configured formatting codec 274. In an 
embodiment, the formatter processor 271 may configure and activate one or more available 
hardware and/or software modules by downloading appropriate software modules to perform 
particular decoding, formatting, and coding tasks as required by a user content download 
request or other content acquisition task. The ability of the formatting codecs 274 to be 
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1 software configurable may allow a high degree of configuration flexibility as well as the 

2 ability to accommodate coding and formatting schemes yet to be developed. 

3 An example of a formatting codec 274 configuration is shown in detail in Figure 8. 

4 The formatting codec 274 includes a master codec processing module 315 that is coupled to 

5 one or more other processing modules and that may manage the configuration and operation 

6 of one or more other processing modules within the formatting codec 274. hi an 

7 embodiment, the processing modules within the formatting codec 274 may be interconnected 

8 using a bus topology that may allow dynamic interconnectivity between up to all of the 

9 processing modules and may support logical addressing of modules for communications 
q 10 routing. The master codec processing module 315 may be coupled to a memory management 
;S 1 1 module 319 and a codec memory module 320. The memory management module 319 may 
%I 12 dynamically assign and manage resources within the codec memory 320 for use by one or 
g 13 more other processing modules within the formatting codec 274. 

J J 14 The master codec processing module 315 is coupled to a content decoding module 

* 15 316 that may manage the configuration and operation of one or more processing modules in 

O 16 order to perform content decoding operations. Coupled to the content decoding module 316 

17 is a frame alignment module 321 that may use decoding parameters received from the 

O 18 formatter processor 271 to synchronize and align the formatting codec 274 with the incoming 

19 content data packets and/or frames. Also coupled to the content decoding module 316 is a 

20 metadata collection module 322 that may read any metadata information contained in one or 

21 more content files and content transport streams and may temporarily store this data in the 

22 codec memory 320. A time code processing module 323 is also coupled to the content 

23 decoding module 316 and may extract any time code data that may be embedded in one or 

24 more content files and/or content transport streams and may store and/or process the time 

25 code as required to support coding and formatting operations. The content decoding module 

26 316 is further coupled to a format translation module 324, which may perform the actual 

27 translation of the incoming source content data to the desired output target content format. 

28 An example of format translation may involve reading a source video file in a proprietary 

29 digital format and translating the content data into a standardized format. The content 

30 decoding module 316 is also coupled to an auxiliary services divider module 328 and an 

31 auxiliary services routing module 329. The auxiliary services divider module 328 may read 
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1 any auxiliary services that may be a part of the source content files and/or transport stream 

2 and may separate, or parse, these data services from the content data and metadata. The 

3 auxiliary services routing module 329 may then route the auxiliary services data to one or 

4 more of the codec memory 320 and/or auxiliary services processors 276. 

5 The master codec processing module 315 is coupled to a content formatting module 

6 317 that may perform content formatting operations and/or manage the configuration and 

7 operation of one or more processing modules in order to perform content formatting 

8 operations. Coupled to the content formatting module 3 17 is a format translation module 324 

9 that may perform translation of incoming source content data into desired output target 
^ 10 content format. Processing steps to accomplish content formatting and format translation 
■0 1 1 may be optimized to minimize degradation of the content data and/or signal. The content 
vj 12 formatting module 317 is also coupled to a digital-to-analog and analog-to-digital conversion 

13 module 325 that may perform any required conversion of audio and video content between 

5 j 14 analog and digital formats to support source content that may exist in various digital and/or 

5 1 5 analog formats and may be delivered to system users in various digital and/or analog formats. 

16 The content formatting module 317 is also coupled to an auxiliary services combiner module 

IK 17 330, which may combine and/or recombine any previously existing and/or newly established 

g 18 auxiliary services with the target content files and/or transport stream. The auxiliary services 

^ 19 combiner 330 is in turn coupled to the auxiliary services routing module 329 in order to 

20 receive appropriate auxiliary services data. The content formatting module 317 is further 

21 coupled to an audio/video processing module 331 that may perform various processing 

22 functions on the audio and video signals of the source content to meet target content 

23 requirements. The audio/video processing functions may include adjusting audio levels, 

24 combining and/or splitting audio signals, and adjusting video luminance and chrominance. 

25 The master codec processing module 315 may also be coupled to a content coding 

26 module 318 that may perform content coding operations and/or manage the configuration and 

27 operation of one or more processing modules in order to perform content coding operations. 

28 Coupled to the content coding module 318 is a digital compression module 326 that, as 

29 required, may apply various digital compression schemes to the target content that was not 

30 previously digitally compressed during content formatting and or translation. Different 

31 levels of digital compression may be applied to one or more target content files and/or 
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1 transport streams. The different compression levels may be based on requirements such as 

2 those regarding content delivery quality and available content delivery bandwidth and/or 

3 storage capacities. An error correction coding module 327 is also coupled to the content 

4 coding module 318 and may apply one or more of various digital forward error correction 

5 coding schemes to one or more of the target content files and/or transport streams to reduce 

6 digital errors during content delivery and allow correction of data containing a certain level 

7 of errors. 

8 The codec and content formatter 270 may contain one or more banks of auxiliary 

9 services processors 276. The auxiliary services processors may comprise one or more 
^ 10 separate software and/or hardware modules. The auxiliary services processors 276 may 
yi| 11 process and/or generate auxiliary services to be included with target content. Such auxiliary 
J1 12 services may include closed captioning services, descriptive video services, advertising, 
•gj 13 interactive services, and other data services. The hardware and/or software modules may 
CP 14 exist as a flexible group of resources that may be accessed on a demand basis and that may 
J* 15 be dynamically assigned and configured to accommodate various auxiliary services 
^ 16 processing operations. When the codec and content formatter processor 271 receives a 

17 coding and formatting request that includes auxiliary services processing requests from the 

5J 18 content acquisition server 400 or the content delivery server 450, the formatter processor 271 

19 may analyze the auxiliary processing parameters contained in the coding and formatting 

20 request. The analyzed parameters are then used to determine the required configuration of 

21 one or more auxiliary services processors 276 necessary to perform the requested operations. 

22 The formatter processor 271 may then poll the group of auxiliary services processing 

23 resources, identify the available resources required for the requested task, configure those 

24 resources to perform the requested task, and transmit the required auxiliary services 

25 parameters as well as routing information to the auxiliary services processor 276. In an 

26 embodiment, the formatter processor 271 may configure and activate one or more available 

27 hardware and/or software modules by downloading appropriate software modules to perform 

28 particular auxiliary services processing tasks as required by a user content download request 

29 or other content acquisition task. The ability of the formatting codecs 274 to be software 

30 configurable allows a high degree of configuration flexibility as well as the ability to 

3 1 accommodate auxiliary services yet to be developed. 
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Auxiliary services may accompany both analog and digital content deliveries and 
may be embedded in one or more content files and/or transport streams. Auxiliary services 
accompanying video programming content may include closed captioning services for users 
with hearing impairments, descriptive video services for system users with visual 
impairments, alternative language audio service, v-chip program rating information, critical 
review information, device control parameters and commands, future programming 
schedules, advertising, advertising insert tags, other text and data services, and interactive 
services data such as browser pages, menus, icons, and links. Auxiliary services 
accompanying audio programming content may include extended metadata such as graphics 
files, in-depth data about the producers of the content, critical reviews, future programming 
schedules, device control parameters and commands, advertising, advertising insert tags, 
other text and data services, and interactive services data such as browser pages, menus, 
icons, and links. Auxiliary services processed by the formatter 270 may be pre-existing 
services recovered from source content and routed from one or more formatting codecs 274 
to one or more auxiliary services processors 276 and/or newly established services 
transferred from one or more other content search, packaging, and delivery system 200 
resources through the codec and content formatter processor 271 to one or more auxiliary 
services processors 276. In an embodiment, one or more auxiliary services processors 276 
may comprise an audio closed captioning processor that may receive audio and time code 
data for a particular video content entity from a formatting codec 274, perform speech- 
recognition and other processing on the audio data, and output one or more files and/or 
transport streams that contain time-code-synchronized closed captioning text and data to be 
combined with the target content as an auxiliary service. 

Figure 9 shows an overview of the content formatting and coding process 360 
performed by the codec and content formatter 270. The codec and content formatter 
processor 271 (see Figure 8) first receives a coding and formatting request from the content 
acquisition server 400 and/or the content delivery server 450, routine 361. The formatter 
processor 271 then uses routine 363 to analyze the coding and formatting request parameters 
contained in the coding and formatting request in order to determine the required 
configuration of one or more formatting codecs 274 necessary to perform the requested 
operations. The formatter processor 271 may then poll formatting and coding resources, 
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1 routine 365, and identify available formatting codec hardware and/or software modules 

2 necessary to realize the required configuration. Routine 367 may then configure one or more 

3 formatting codecs 274 using the coding and formatting request parameters and required 

4 configuration data to access and configure one or more formatting codec 274 hardware 

5 and/or software modules. The formatter processor 271 may then use source content and 

6 target content routing address information from a decoding and formatting request to 

7 configure one or more content routers 272, routine 369. Routine 371 may then initiate and 

8 manage the transfer of source content through one or more content routers 272 from one or 

9 more source locations to one or more formatting codecs 274. Source locations may include 

10 the content acquisition server 400, the content delivery server 450, aggregator local storage 

1 1 254, and local streaming sources 262. One or more formatting codecs 274 may then execute 

12 routine 373, which may make use of the hardware and/or software modules shown in detail 

13 in Figure 8 to perform the requested decoding, formatting, and coding operations to produce 

14 the target output content. Routine 374 may then perform processing of auxiliary services as 

15 outlined in the content coding and formatting request. Auxiliary services processing starts at 

16 the formatting codec 274, which parses existing auxiliary services from the source content 

17 and routes those services to one or more auxiliary services processors 276, which perform 

18 appropriate processing on existing auxiliary services as well as establish new auxiliary 

19 services as required. The auxiliary services processors 276 may then route the appropriate 

20 auxiliary services to one or more formatting codecs 274 for combination with one or more 

21 target output content files and/or data transport streams. 

22 Once the one or more target content files and/or transports streams have been 

23 assembled, routine 375 within the formatter processor 271 transfers the formatted target 

24 content through one or more content routers 272 to one or more target addresses residing in 

25 one or more components external to the codec and content formatter 270 which may include 

26 the content acquisition server 400, the content delivery server 450, aggregator local storage 

27 254, and/or local streaming sources 262. The formatter processor 271 may then clear content 

28 routing connections using routine 377. Content routing connections to be cleared may 

29 include one or more physical routing connections and/or one or more logical routing 

30 connections. Physical routing connections may be cleared by returning routing circuitry to 

31 an idle or standby state and logical routing connections may be cleared by removing logical 
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1 address information from appropriate memory registers within one or more content routers 

2 272. The formatter processor 271 may then return one or more configured formatting codecs 

3 274 and/or auxiliary services processors 276 to a standby and/or default state, routine 379, 

4 and return the one or more formatting codec 274 and or auxiliary services processor 276 

5 hardware and/or software modules to the available pools of formatting resources and/or 

6 auxiliary services processing resources, respectively. Routine 381 within the formatter 

7 processor 271 may then log appropriate administrative data with the content acquisition 

8 server 400 and/or the content delivery server 450. Routine 383 may then stop processing. 

9 Figure 10 shows the auxiliary services processing of routine 374 in more detail. If 
10 existing auxiliary services and/or newly established auxiliary services are to be combined 

m 11 with target output content, then routine 385 within one or more of the formatter processor 

^1 12 271 and/or formatting codecs 274 analyzes any auxiliary services processing request 

|g 13 parameters that may be part of the coding and formatting request and determines the 

01 14 necessary configuration of one or more auxiliary services processors 276. Routine 387 

^ " 1 5 within the formatter processor 271 may then poll available auxiliary processing resources and 

H*-. 16 configure one or more auxiliary services processors 276 required to perform the requested 

O 

P 17 processing operations. One or more formatting codecs 274 may parse, or separate, existing 

S 18 auxiliary services and time code data that may be combined with one or more source content 

19 files and/or transport streams, routine 389. Routine 391 may then transfer these auxiliary 

20 services as well as the source content time code data to one or more auxiliary services 

21 processors 276. Routine 391 may also transfer data from sources external to the codec and 

22 content formatter 270 that may be required to establish new auxiliary services to be 

23 combined with the target output content. These external sources may include the content 

24 acquisition server 400, the content delivery server 450, and the aggregator local storage 254. 

25 One or more auxiliary services processors 276 may then perform one or more processing and 

26 formatting operations on one or more sources of auxiliary services data in accordance with 

27 processing request parameters in order to output one or more target auxiliary services to be 

28 combined with target output content, routine 393. Routine 395, which may reside within the 

29 formatter processor 271 and/or one or more auxiliary services processors 276, may then 

30 transmit these output auxiliary services to one or more formatting codecs 274. Routine 397 

31 may then be executed by one or more formatting codecs 274 to combine the auxiliary 
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services with one or more target output content files and/or data streams. Processing is then 
continued with routine 375 as shown in Figure 9 and as described above. 
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